Image processing device, non-transitory computer readable medium, and image processing method

ABSTRACT

An apparatus is configured to: cause a display to display a GUI comprising a plurality of items; receive user input that is scribed over any of the plurality of items displayed on the display; provide the user with a substantially real-time feedback of the scribing by applying digital ink indicating a scribed line on the GUI; select a first item of the plurality of items based on a position of the scribing within the GUI; recognize a trail of the scribed line; in response to the trail of the scribed line being recognized as a first predetermined trail, perform, on the first item, first processing corresponding to the first predetermined trail; and in response to the trail of the scribed line being recognized as a second predetermined trail, perform second processing on the first item. The second processing corresponds to the second predetermined trail and is from the first processing.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a Division of application Ser. No. 14/796,291 filed Jul. 10,2015, which in turn is a Continuation of application Ser. No. 13/606,545filed Sep. 7, 2012, which is based on and claims priority under 35 USC119 from Japanese Patent Application No. 2012-043010 filed Feb. 29,2012. The disclosure of the prior applications is hereby incorporated byreference herein in its entirety.

BACKGROUND Technical Field

The invention relates to an image processing device, a non-transitorycomputer readable medium, and an image processing method.

SUMMARY

According to an aspect of the invention, there is provided aninformation processing apparatus comprising a processor configured to:cause a display to display a graphical user interface comprising aplurality of items; receive an input from a user, the input beingscribing over any of the plurality of items displayed on the display;provide the user with a substantially real-time feedback of the scribingby applying digital ink indicating a scribed line on the graphical userinterface; select a first item of the plurality of items based on aposition of the scribing within the graphical user interface; recognizea trail of the scribed line; in response to the trail of the scribedline being recognized as a first predetermined trail, perform firstprocessing on the first item, the first processing corresponding to thefirst predetermined trail; and in response to the trail of the scribedline being recognized as a second predetermined trail, perform secondprocessing on the first item, the second processing corresponding to thesecond predetermined trail and being different from the firstprocessing.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiments of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a block configuration diagram illustrating an exemplaryembodiment of an image processing device according to the invention;

FIG. 2 is a hardware configuration diagram of an image processing deviceaccording to a first exemplary embodiment;

FIG. 3 is a diagram illustrating an example of function setting patterninformation stored in a function setting pattern storage unit of thefirst exemplary embodiment;

FIGS. 4A and 4B are diagrams illustrating an example of a functionsetting screen displayed on an operation panel of the first exemplaryembodiment;

FIGS. 5A and 5B are diagrams illustrating another example of a functionsetting screen displayed on the operation panel of the first exemplaryembodiment;

FIGS. 6A and 6B are diagrams illustrating an example of transition of adisplay content of the function setting screen displayed on theoperation panel of the first exemplary embodiment;

FIG. 7 is a flowchart illustrating a function setting process of thefirst exemplary embodiment;

FIGS. 8A and 8B are diagrams illustrating an example of a functionsetting screen displayed on an operation panel of a second exemplaryembodiment;

FIG. 9 is a flowchart illustrating a function setting process of thesecond exemplary embodiment;

FIG. 10 is a diagram illustrating an example of a setting contentconfirmation screen used in the second exemplary embodiment;

FIGS. 11A to 11D are diagrams illustrating a part of a function settingscreen displayed on an operation panel of a third exemplary embodiment;

FIG. 12 is a flowchart illustrating a function setting process of thethird exemplary embodiment;

FIG. 13 is a flowchart illustrating a function setting process of afourth exemplary embodiment; and

FIGS. 14A to 14C are diagrams illustrating a part of a function settingscreen displayed on an operation panel of a sixth exemplary embodiment.

DETAILED DESCRIPTION

Hereinafter, an exemplary embodiment of the invention will be describedwith reference to the drawings.

First Exemplary Embodiment

FIG. 1 is a block configuration diagram illustrating an exemplaryembodiment of an image processing device according to the exemplaryembodiment of the invention. Constituent components which are notnecessary for description of this exemplary embodiment will not beillustrated. FIG. 2 is a hardware configuration diagram of an imageprocessing device 10 according to this exemplary embodiment. The imageprocessing device 10 is a multi-function machine that executes varioustypes of image processing such as copying, printing, and scanning and isa device in which a computer is included.

In FIG. 2, a CPU 21 controls the operations of various mechanismsmounted on this device such as a scanner 24 and a printer engine 26according to a program stored in a ROM 29. An address data bus 22 isconnected to the various mechanisms that serve as a control target ofthe CPU 21 and performs data communication. An operation panel 23 is auser interface device that receives instructions from a user anddisplays information. The scanner 24 reads a document set by the userand stores the read document in a hard disk drive (HDD) 25 or the likeas electronic data. The HDD 25 stores the electronic document read usingthe scanner 24. The printer engine 26 prints an image on an output sheetaccording to an instruction from a control program that is executed bythe CPU 21. A network interface (I/F) 27 is connected to a network 31and is used for transmitting electronic data generated by this device,receiving an electronic mail sent to this device, and accessing thisdevice via a browser. A RAM 28 is used as a working memory duringexecution of a program and a communication buffer during transmissionand reception of electronic data. The ROM 29 stores various types ofprograms associated with image processing executed by this device, thecontrol of the processing, encryption of electronic data, andtransmission and reception of the electronic data. When various types ofprograms are executed, the respective constituent components describedlater perform predetermined processing functions. An external mediainterface (I/F) 30 is an interface to an external memory device such asa USB memory or a flash memory.

Returning to FIG. 1, the image processing device 10 of this exemplaryembodiment includes a user interface (UI) unit 11, a gesture recognitionunit 12, a function determining unit 13, a parameter identification unit14, a parameter setting unit 15, an image processing unit 16, a controlunit 17, a function setting pattern storage unit 18, and a settinginformation storage unit 19. The user interface (UI) unit 11 is a userinterface unit that is realized in coordination with the operation panel23 and includes an operation receiving unit 111 and a display processingunit 112. The display processing unit 112 is a display unit thatdisplays a setting screen for functions that may be set when selectingimage processing and executing selected image processing. The operationreceiving unit 111 receives the operation of the user on the screendisplayed on the operation panel 23. The user inputs settings by tracingon a display screen of the operation panel 23 with a finger or a pen,and in this exemplary embodiment, the trail of the operation of the usertracing on the display screen will be also referred to as a “gesture.”

The gesture recognition unit 12 is provided as a recognition unit andrecognizes the trail, namely a gesture, of the operation of the user onthe function setting screen displayed on the operation panel 23. Thefunction determining unit 13 is provided as a determining unit anddetermines a function selected by the user based on the position of thetrail of the operation recognized by the gesture recognition unit 12.The parameter identification unit 14 is provided as an identificationunit and identifies a parameter of the function set by the user withrespect to the function determined by the function determining unit 13from the trail of the operation recognized by the gesture recognitionunit 12. Here, the “parameter” is an operation condition necessary foroperating a function, and values, positions, and the like are set as theparameter. For example, a character string representing a sheet size(A4, A3, or the like) is set as the parameter for a sheet size settingfunction, and a numerical value such as a tray number (1, 2, 3, or thelike) is set as the parameter for a sheet tray selecting function.

The parameter setting unit 15 is provided as a setting unit and performssettings for executing image processing with a combination of thefunction determined by the function determining unit 13 and theparameter identified by the parameter identification unit 14. Here, the“image processing” in this exemplary embodiment is image processingexecuted in the image processing device 10. For example, copying,printing, faxing, or the like corresponds to the image processing inthis exemplary embodiment. Thus, in executing image processing calledcopying, for example, the parameter setting unit 15 sets “A4” as theparameter for a sheet size selecting function of the copying functionand sets “double-side” as the parameter for a both-side printingfunction according to the gesture of the user. The image processing unit16 executes image processing according to the content set by theparameter setting unit 15. The control unit 17 controls the execution ofimage processing in coordination with the constituent components 11 to16.

In this exemplary embodiment, the parameters of the respective functionsare set according to the gesture of the user, and the parameters set tothe functions are stored in the setting information storage unit 19.Further, parameters (default values) that are initially set to therespective functions are stored.

FIG. 3 is a diagram illustrating an example of function setting patterninformation stored in the function setting pattern storage unit 18 inthis exemplary embodiment. The types of the parameters set to therespective functions, the conditions, and the like are set in thefunction setting pattern information. In the function setting patterninformation of this exemplary embodiment, respective items ofinformation such as a first gesture, a function position, a subsequentgesture, and a setting content are correlated with the respectivefunctions. In the function, a function provided in this exemplaryembodiment is set. In the first gesture, a parameter that is to be setinitially for the corresponding function and a description of theparameter that is to be set are described. For example, a numericalvalue is set for a magnification and a flag value (0/1) is set for acolor mode as the parameter. In the function position, “O” is set for afunction that requires designation of a position as the parameter. Forexample, a single-side/double-side function requires setting of pluralparameters on whether a document is single-sided or double-sided andwhether single-sided printing or double-sided printing will beperformed. Thus, only the setting of whether a document is single-sidedor double-sided (in this exemplary embodiment, “the first gesture”) isnot sufficient, and it is necessary to set whether a copy (copyingsheet) is single-sided or double-sided together as an additionalparameter subsequently to the first gesture. In this exemplaryembodiment, the gesture for setting a parameter that needs to be settogether with the parameter set by the first gesture will be referred toas a “subsequent gesture.” Moreover, in this exemplary embodiment, aparameter set by the subsequent gesture will be referred to as a“subsequent parameter.” A required subsequent gesture may be changedaccording to the parameter set by the first gesture as in the aggregatedoutput function. In the setting content, descriptive text for describingrespective gestures is described as reference information.

The respective constituent components 11 to 17 of the image processingdevice 10 are realized by coordination of the computer included in theimage processing device 10 and the program operated by the CPU 21mounted on the computer. Moreover, the respective storage units 18 and19 are realized by the HDD 25 or the RAM 28 mounted on the imageprocessing device 10. Alternatively, the storage units 18 and 19 may berealized using an external storage device that may be accessed via thenetwork IF 27.

Moreover, the program used in this exemplary embodiment may be providedby a communication unit and may be provided by being stored in acomputer-readable recording medium such as a CD-ROM or a DVD-ROM. Theprogram provided from the communication unit or the recording medium isinstalled in a computer, and the CPU of the computer executes theprogram sequentially, whereby various types of processing are realized.

Next, the operation of this exemplary embodiment will be described.First, the operation of this exemplary embodiment will be described fromthe perspective of the user of the image processing device 10 withreference to FIGS. 4A and 4B and FIGS. 5A and 5B.

FIGS. 4A and 4B illustrate an example in which functions (a sheet traysetting function and functions B to D) are displayed on the operationpanel 23 as a function that may be selected for image processing calledcopying. A function display (region) corresponding to each function isformed by a graphical user interface (GUI) component (function selectingcomponent) for selecting the corresponding function.

As illustrated in FIG. 4A, referring to a function display region 41 ofa sheet tray, “tray 1” is currently set as a sheet tray. Here, it isassumed that the user wants to change the sheet tray to “tray 2.” In therelated art, the user may select a function button for selecting a sheettray to display a setting screen for a sheet tray selecting function onthe operation panel and select “tray 2” on the setting screen. In thisexemplary embodiment, the user may write a numerical value (in thisexample, “2”) that the user wants to set and change within a targetfunction display region. More specifically, the user inputs a settingvalue to be changed by a trail 42 of an operation for writing “2” sothat a starting point 43 of the trail 42 is inside the function displayregion for the sheet tray. As a result, in this exemplary embodiment, asin the display example of the function display region 41 illustrated inFIG. 4B, the parameter of the sheet tray is set and changed to “tray 2.”

Moreover, as illustrated in FIG. 5A, referring to a function displayregion 44 which represents that image processing for copying is selectedand which represents the number of copies, “1” is currently set as thenumber of copies. Here, the user wants to copy “three” copies. In thiscase, the user writes “3” in the target function display region 44 sothat a starting point 46 of a trail 45 of the operation for writing “3”is inside the function display region 44. As a result, the number ofcopies is set to “3” as in the display example of the function displayregion 44 illustrated in FIG. 5B.

A function setting process of setting and changing parameters accordingto the gesture will be described from the perspective of the operationof the image processing device 10 with reference to a screen displayexample illustrated in FIGS. 6A and 6B and the flowchart illustrated inFIG. 7.

The image processing device 10 causes the display processing unit 112 todisplay a function setting screen illustrated in FIG. 6A on theoperation panel 23 according to a predetermined operation and performsstandby until the user starts inputting an operation (step S101: N).Here, it is assumed that the user started inputting an operation with astarting point located within a function display region 47 of thefunction A. Although the operation is performed with a finger tip, atool such as a pen may be used. When the operation receiving unit 111detects the start of the operation (step S101: Y) and recognizes thatthe starting point of the operation is present within any one of thefunction display regions (step S102: Y), the display processing unit 112highlights the function display region (step S103). When the startingpoint of the operation is not included in any one of the functiondisplay regions (step S102: N), the process returns to the beginning.FIG. 6B illustrates a display example of a function setting screen inwhich a starting point of a gesture 48 is included in the functiondisplay region 47 correlated with the function A, and the functiondisplay region 47 is highlighted. In this exemplary embodiment, thedisplay form of the selected function display region 47 is highlightedby thickening the border line so that the display form is different fromthe display forms of the other function display regions. In this way,the selected function display region 47 may be easily perceived. Amethod of making the display forms different is not limited to this. Forexample, the type of border line or a display color may be madedifferent, or the border line may flash. Alternatively, the displayother than the selected function display region 47 may be shaded.

Here, when the end of the operation is recognized from the finger tipbeing separated from the operation panel 23 (step S104: Y), the displayprocessing unit 112 stops the highlighting of the function displayregion 47 (step S105).

When the gesture recognition unit 12 recognizes the trail of the aboveoperation, that is, the gesture, the function determining unit 13determines a function that serves as a setting target of the parameterset by the gesture of the user from the gesture (step S106). Morespecifically, since the screen image displayed on the operation panel 23and the display position information of the respective function displayregions included in the screen image are known in advance, by comparingcoordinate data that specifies the display positions of the respectivefunction display regions with coordinate data of the starting point ofthe gesture, the function that serves as a setting target of theparameter is specified.

Subsequently, the parameter identification unit 14 identifies aparameter set to the function specified by the function determining unit13 from the gesture recognized by the gesture recognition unit 12 (stepS107). Parameters “2” and “3” are identified in the examples illustratedin FIGS. 4A and 5A, respectively. Although the parameter identificationunit 14 may identify a character designated by the gesture by acharacter recognition process only, in this exemplary embodiment, sincethe type of parameters that may be set to the respective functions areregistered in the function setting pattern storage unit 18, recognitionaccuracy may be improved by referring to the function setting patternregistered in the function setting pattern storage unit 18.Alternatively, by comparing the function setting pattern registered inthe function setting pattern storage unit 18, in particular the firstgesture with the character identified for the gesture through thecharacter recognition process, it may be determined whether theoperation of the user is a valid gesture for the selected function.

In this way, when the function selected by the user and the parameter ofthe function are set, the parameter setting unit 15 reflects the setparameter in the function selected by the user. In response to this, thecontrol unit 17 causes the display processing unit 112 to update thedisplay of the function display region corresponding to the functionselected by the user with the content in which the set parameter isreflected (step S108). The user is informed of the fact that the setparameter is reflected by changing the display within the functiondisplay region 41 to “tray 2” based on “2” identified from the gesturein the example illustrated in FIG. 4B and changing the display withinthe function display region 44 to “3” based on “3” identified from thegesture in the example illustrated in FIG. 5B. Moreover, when aninstruction to execute image processing is received from the userpressing an execute button (not illustrated), the image processing unit16 executes image processing according to the parameters set to therespective functions.

Second Exemplary Embodiment

In the first embodiment, the function that serves as a setting target ofthe parameter is specified by the position of the starting point of thegesture. In this exemplary embodiment, rather than the starting point ofthe gesture, the function correlated with a function display region inwhich a largest amount of the gesture is included among the functiondisplay regions displayed on the function setting screen is determinedas the function selected by the user.

Hereinafter, the operation of this exemplary embodiment is described.First, the operation of this exemplary embodiment will be described fromthe perspective of the user of the image processing device 10 withreference to FIG. 8.

FIGS. 8A and 8B illustrate the same function setting screen as FIGS. 4Aand 4B described in the first exemplary embodiment. As illustrated inFIG. 8A, referring to the function display region 41 of the sheet tray,“tray 1” is currently set as a sheet tray. Here, it is assumed that theuser wants to change the sheet tray to “tray 2.” In this exemplaryembodiment, the user may write a numerical value (in this example, “2”)that the user wants to set and change within a target function displayregion. More specifically, the user performs an operation so that alargest amount of the trail 42 of the operation of writing a parameter“2” to be changed are included in the function display region 41 of thesheet tray. As a result, in this exemplary embodiment, the parameter ofthe sheet tray is set and changed to “tray 2” as in the display exampleof the function display region 41 illustrated in FIG. 8B.

A function setting process of setting and changing parameters accordingto a gesture will be described from the perspective of the operation ofthe image processing device 10 with reference to the flowchartillustrated in FIG. 9.

The image processing device 10 causes the display processing unit 112 todisplay a function setting screen illustrated in FIG. 8B on theoperation panel 23 according to a predetermined operation and causes theuser to perform a parameter setting operation. When the operationreceiving unit 111 receives a gesture of the user performed on theoperation panel 23 (step S201), the gesture recognition unit recognizesthe gesture. Subsequently, the function determining unit 13 determines afunction correlated with a function display region in which a largestamount of the gesture is included among the function display regions inwhich the gesture is included by referring to the gesture as a functionthat serves as a setting target of the parameter (step S202). Morespecifically, since the screen image displayed on the operation panel 23and the display position information of the respective function displayregions included in the screen image are known in advance, the length ofthe line of the gesture present within a display range of each of thefunction display regions is calculated, and the calculated line lengthsare compared. In the example illustrated in FIG. 8A, although thegesture is also included in the function display region 41 and thefunction display region 42, since a larger amount of the gesture isincluded in the function display region 41, in this example, thefunction correlated with the function display region 41 is determined asthe function that serves as a setting target of the parameter.

Subsequently, the parameter identification unit 14 identifies aparameter set to the function specified by the function determining unit13 from the gesture recognized by the gesture recognition unit 12 (stepS203). A parameter “2” is identified in the example illustrated in FIG.8A. The process performed by the parameter identification unit 14 is thesame as that of the first exemplary embodiment.

In this way, when the function selected by the user and the parameter ofthe function are set, the parameter setting unit 15 reflects the setparameter in the function selected by the user. The setting contentbefore this updating is written to and stored in the setting informationstorage unit 19. The control unit 17 informs the user of the change inthe setting by causing the display processing unit 112 to change thedisplay form of the function display region 41 that serves as a settingtarget so as to be different from the display forms of the otherfunction display regions. In this exemplary embodiment, the functiondisplay region 41 is highlighted (step S204). In this case, theparameter displayed within the function display region 41 is set to“tray 2.” Subsequently, the control unit 17 sets a timer in order toprovide a time to cancel the above operation of the user (step S205).

Here, the user perceives that the setting is not for a desired functionor the parameter is not a desired setting value even if the setting isfor the desired function by referring to the highlighted functiondisplay region 41 and performs a predetermined cancel operation. Thatis, before a timeout occurs (step S206: N), when the user performs acancel operation (step S207: Y), the parameter setting unit 15 reads animmediately previous setting content stored in the setting informationstorage unit 19 and updates the setting with the read setting content.In this way, the above-described setting content is canceled andreturned to the original setting content. In response to this, thecontrol unit 17 causes the display processing unit 112 to stop thehighlighting of the function display region 41 and to perform display ina state where the setting content is returned to the original settingcontent (step S208).

On the other hand, when the cancel operation is not performed until thetimeout occurs (step S206: Y), the control unit 17 causes the displayprocessing unit 112 to stop the highlighting of the function displayregion 41 (step S209). Moreover, in response to the timeout, theabove-described setting content is confirmed. That is, the settingscreen of the sheet tray is updated as illustrated in FIG. 8B.

As an example of the cancel operation of the user in the above-describedprocess, after the highlighting in step S204, the user performs anoperation of pressing a cancel button (not illustrated) displayed on thescreen or performs an operation of inputting a predetermined gesture forcanceling the setting.

However, in the above description, the user inputs confirmation in astate where the content of the gesture is reflected in the functiondisplay region 41. Moreover, when a desired function is not selected,the user has to cancel the setting. However, as illustrated in FIG. 10,a confirmation screen 50 that displays the setting content may bedisplayed as a pop-up window. That is, rather than reflecting thesetting content in the function display region 41, the confirmationscreen 50 is displayed so that the function determined to be selected bythe user may be perceived. In FIG. 10, the confirmation screen 50 isdisplayed so as to be superimposed on the function display region 41corresponding to the function that is determined to be selected. Theconfirmation screen 50 includes a region 51 where the gesture isdisplayed, an OK button 52 that is selected when a combination of thegesture and the function that serves as a setting content correspondingto the gesture is correct, and a cancel button 53 that is selected whenthe user wants to cancel the setting since the combination is notcorrect. Thus, the user selects any one of the OK button 52 or thecancel button 53 according to a confirmation command using theconfirmation screen 50.

Moreover, the confirmation screen 50 is displayed so as to be movable onthe screen, and if the setting is made for a function correlated withthe other function display region, the user may move the confirmationscreen 50 to a function display region corresponding to the properfunction by a drag-and-drop operation, for example so that the movingoperation is recognized as confirmation or may select the OK button 52at the destination. For example, if the gesture “2” is intended to beset as a parameter of the number of copies, the user changes the settingto the setting for a copy count setting function by moving theconfirmation screen 50 from the function display region 41 to a functiondisplay region for the copy count setting function. In this way, thecancel operation of step S207 of FIG. 9 may be not performed.

In this exemplary embodiment, the above-described pop-up window isdisplayed so as to be superimposed on the function display region 41 sothat the function selected by the user may be identified, and thefunction may be changed so as to be correlated with a proper function ifthe function selected by the user is misperceived. It is not necessaryto use the pop-up screen, and the corresponding function may be changedusing another method.

Third Exemplary Embodiment

Depending on a parameter, there is a parameter which requires anadditional parameter to be set subsequently to the set parameter. FIGS.11A to 11D illustrate an example of such a case. FIGS. 11A to 11D arediagrams illustrating an example of a function display regioncorresponding to a function (hereinafter referred to as an “aggregatedoutput function”) of outputting plural pages on one sheet in anaggregated manner. According to the display example of the functiondisplay region 53 illustrated in FIG. 11A, a gesture for outputting fourpages on one sheet together is input in a state where it is set not touse the aggregated output function. Here, when four pages are output onone sheet together (hereinafter referred to as “4up”), there are fourpatterns for arranging the pages, which include a pattern (hereinafterreferred to as “Z”) of top-left, top-right, bottom-left, andbottom-right, a pattern (hereinafter referred to as “inverted-Z”) oftop-right, top-left, bottom-right, and bottom-left, a pattern(hereinafter referred to as “inverted-N”) of top-left, bottom-left,top-right, and bottom-right, and a pattern (hereinafter referred to as“N”) of bottom-left, top-left, bottom-right, and top-right. Thus, when4up is set as a parameter, it is necessary to set any one of theparameters of Z, inverted-Z, inverted-N, and N as a pattern forarranging the pages.

Thus, in this exemplary embodiment, when it is necessary to set anaddition parameter depending on a parameter that is first set, thefollowing operation is performed. First, the operation of this exemplaryembodiment will be described from the perspective of the user of theimage processing device 10 with reference to FIGS. 11A to 11D.

As illustrated in FIG. 11A, “4” is designated in a function displayregion 53 corresponding to the aggregated output function by a gesture54. In the case of 4up, as described above, since there are fourpatterns for arranging the pages, a guide display screen 55 for settingthe parameter (the “subsequent gesture” described in the first exemplaryembodiment in FIG. 3) is displayed so as to be correlated with afunction display region 53. In FIG. 11B, the guide display screen 55 issuperimposed on the function display region 53 as an example of thecorrelation. Here, the user performs an operation of specifying any oneof the parameters of Z, inverted-Z, inverted-N, and N as a pattern forarranging the pages by referring to the guide display screen 55. FIG.11C illustrates an example of a gesture 56 of setting Z. As a result, asin the function display region 53 illustrated in FIG. 11D, 4up and anarrangement pattern of Z are set to the aggregated output function.

A function setting process of setting and changing parameters accordingto the gesture will be described from the perspective of the operationof the image processing device 10 with reference to the flowchartillustrated in FIG. 12.

The image processing device 10 causes the display processing unit 112 todisplay a function setting screen including the function display region53 illustrated in FIG. 11A on the operation panel 23 according to apredetermined operation and causes the user to perform a parametersetting operation. When the operation receiving unit 111 receives agesture of the user performed on the operation panel 23 (step S301), thegesture recognition unit 12 recognizes the gesture. Subsequently, thefunction determining unit 13 determines a function that serves as asetting target of the parameter by referring to the gesture andidentifies the parameter to be set to the function to thereby confirmthe function and the parameter (step S302). The determination of thefunction and the identification of the parameter in step S302 may beperformed according to the method illustrated in the first or secondexemplary embodiment. Thus, detailed description thereof will not beprovided. By this process, according to the example illustrated in FIG.11A, the first parameter “4” is identified by the first gesture.

Subsequently, the control unit 17 determines whether a subsequentparameter is required. That is, the control unit 17 refers to thesetting content of the subsequent gesture of the function settingpattern information stored in the function setting pattern storage unit18. Referring to the function setting pattern information illustrated inFIG. 3, it may be understood that in the case of the aggregated outputfunction, when the first gesture is 3 or more, it is necessary to setany one of the parameters of Z, inverted-Z, inverted-N, and N as asubsequent parameter. Thus, when it is determined that it is necessaryto set a subsequent parameter (step S303: Y), the control unit 17 causesthe display processing unit 112 to display the guide display screen 55illustrated in FIG. 11B so as to be correlated with the function displayregion 53 as necessary based on the setting content of the subsequentgesture. Referring to the function setting pattern information, in thecase of the aggregated output function or the single-side/double-sidecopying function, it is preferable in terms of the user to generate andoutput the guide display screen in order to present an index as to whichgesture should be performed. However, in the case of a binding functionor a punching function, since the subsequent parameter is a number, itis not necessary to output the guide display screen 55. Subsequently,the control unit 17 sets a timer in order to provide a time to stop theabove operation of the user (step S304).

Here, when the user does not perform an operation different from thefirst gesture, that is, an operation for setting a subsequent parameterbefore a timeout occurs (step S307: N, step S305: Y), and when the userperforms an operation of stopping the operation (step S306: Y) evenbefore the timeout occurs (step S305: N), the setting process for theaggregated output function stops. An example of an operation forstopping the operation includes an operation of selecting an operationstopping button displayed on the screen and an operation of the useradding a predetermined gesture that means stopping of the operation tothe gesture. On the other hand, when the user performs an operation ofsetting a subsequent parameter before the timeout occurs (step S307: Y),the subsequent parameter set by the subsequent gesture is held as aparameter that is additionally set for the target function in additionto the first parameter set by the first gesture (step S308). By thisprocess, according to the example illustrated in FIG. 11C, a subsequentparameter “Z” is identified by the subsequent gesture.

Subsequently, the process returns to step S303, and it is determinedwhether an additional subsequent parameter is required. However, in theexample illustrated in FIGS. 11A to 11D, the subsequent parameter is notrequired. When an additional subsequent parameter is required, theabove-described processes (steps S304 to S308) are repeatedly performed.When the subsequent parameter is not required (step S303: N), theparameter setting unit 15 confirms all parameters (“4up” and thearrangement pattern of “Z”) set for the function (in this example, theaggregated output function) selected by the user (step S309). Inresponse to this, the control unit 17 causes the display processing unit112 to update the display of the function display region correspondingto the function selected by the user with the setting content in whichthe setting parameters are reflected (step S310). FIG. 11D illustrates adisplay example of this case.

As described above, the above-described processes may be performed whenit is necessary to set an additional parameter for the parameter that isfirst set.

Fourth Exemplary Embodiment

Depending on a function provided by image processing, the number ofreceived characters (digits) may be determined according to theintention of the user. For example, in a copy count setting functioncapable of setting the number of copies in the range of 1 to 999, whenan input operation of “1” and “0,” for example, is performed, it is notclear whether the user intends to designate 10 copies or the userintends to continue an operation of “0” to “9” to designate 100 to 109copies at the point in time when the operation corresponding to twodigits is performed. As above, although it is possible to confirm theinput when the user performs an operation that represents a three-digitnumber, it may be desirable to provide a certain determination criterionfor confirming the input operation even when an operation correspondingto one digit or two digits is input.

Therefore, in this exemplary embodiment, it is determined that anoperation for setting a parameter at the point in time when apredetermined period has elapsed from the last operation is an endingoperation, that is the end of the operation is determined according tothe timeout. Moreover, when the user wants to confirm the inputoperation without reaching the upper-limit digit, the user may input apredetermined gesture that means the end of the operation.

FIG. 13 is a flowchart illustrating a process of setting and changingparameters according to this exemplary embodiment. Since the processaccording to this exemplary embodiment is basically the same as that ofthe third exemplary embodiment (FIG. 12), the same processes as those ofFIG. 12 will be denoted by the same step numbers, and redundantdescription will not be provided appropriately. In this exemplaryembodiment, step S401 is added to the processing content of the thirdexemplary embodiment. Specifically, a function that serves as a settingtarget of the parameter and a parameter to be set to the function areconfirmed from the first gesture (step S302). As a result, when it isdetermined that setting of a subsequent parameter is required (stepS303: Y), the timer is set (step S304). When the user performs a certainoperation before a timeout occurs, and the parameter identification unit14 determines that the gesture is a gesture that means ending of theoperation (step S401: Y), the parameter setting unit 15 confirms allparameters based on the gestures performed immediately before thegesture that means the ending of the detected operation (step S309). Inresponse to this, the control unit 17 causes the display processing unit112 to update the display of the function display region correspondingto the function selected by the user with the setting content in whichthe set parameters are reflected (step S310).

According to the above example in which the user wants to set the numberof copies to 10 copies in a copy count setting function capable ofsetting the number of copies in the range of 1 to 999, since a numericalvalue corresponding to the upper-limit third digit is not input at thepoint in time when the user inputs the gesture of “1,” it is determinedthat a subsequent gesture is required (step S303: Y). Moreover, when agesture (that is, the gesture of “0”) that designates a subsequentparameter before the occurrence of the timeout is input (step S307: Y),since the numerical value corresponding to the upper-limit third digitis not input at that point in time, it is determined that the subsequentgesture is required (step S303: Y). Moreover, when the user inputs agesture that means the ending of the operation before the occurrence ofthe timeout (step S401: Y), the setting is finalized for the copy countsetting function into the content that the number of copies is 10 copies(step S309).

Fifth Exemplary Embodiment

A gesture for canceling the parameter set by the gesture, a gesture forstopping the operation, and a gesture for confirming the operation areset in advance respectively in the second, third, and fourth exemplaryembodiments. In this way, when the user inputs an operationcorresponding to each of the gestures, it may be recognized that aninstruction to cancel the set parameter, stop the operation, or confirmthe operation is received. However, the invention is not limited tothis, and a gesture for returning the set parameter to the initialsetting may be set in advance.

FIG. 3 illustrates a gesture “Λ” corresponding to a gesture forreturning the setting to the initial setting. The parameteridentification unit 14 compares the operation performed by the user withthe gesture for returning the setting to the initial setting included inthe function setting pattern information. When the gesture for returningthe setting to the initial setting is detected, the parameter settingunit 15 reads an initial value of the parameter for the function storedin the setting information storage unit 19, for example, and set theread initial value to thereby return the setting for executing imageprocessing to the initial state.

In this exemplary embodiment, although the user may return the parameterto be set for only the functions, in which a gesture for returning thesetting to the initial setting is written, to the initial setting, agesture for returning parameters for all functions to the initial statesuch as a gesture corresponding to a reset button may be provided.

Sixth Exemplary Embodiment

For example, in the case of a binding function or a punching function,when the function is selected, it is necessary to set the position andthe number of positions at which the sheet is bound or punched as aparameter. This exemplary embodiment may deal with such a function.

FIGS. 14A to 14C are diagrams illustrating an example of a functiondisplay region of a binding function. In this exemplary embodiment,although it is assumed that the function display region is displayed onthe function setting screen illustrated in FIGS. 4A and 4B or the like,the function display region may be displayed on the operation panel 23when a binding function is selected on the function setting screenillustrated in FIGS. 4A and 4B.

FIGS. 14A to 14C illustrate a function display region of a bindingfunction, which resembles a sheet. As illustrated in FIG. 14A, thefunction display region 57 is segmented into 8 areas in total whichincludes left, right, top, and bottom areas and the four corner areas,as positions at which the sheet may be bound by a stapler. That is, theuser may select a sheet binding position from eight positions.

In the function display region 57, the user performs an operation at aposition that the user wants to designate as a binding position. FIG.14B illustrates a gesture 59 that involves an operation of drawing aline within an area 58 that represents the left “L.” Moreover, FIG. 14Cillustrates a gesture 60 of “2” that involves a writing operation withthe function display region 57. That is, the user designates the leftend as the binding position at the position of the first gesture 59 asillustrated in FIG. 14B and designates “2” as the number of bindingpositions with the gesture 60 subsequent to the first gesture 59 asillustrated in FIG. 14C. The process of this exemplary embodiment may bethe same as that of the third exemplary embodiment (FIG. 12), andredundant description will not be provided.

Although a method of setting parameters of a characteristic function hasbeen described in the respective exemplary embodiments, thecharacteristic points described in the respective exemplary embodimentsmay be combined appropriately.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. An information processing apparatus comprising aprocessor configured to: cause a display to display a graphical userinterface comprising a plurality of items; receive an input from a user,the input being scribing over any of the plurality of items displayed onthe display; provide the user with a substantially real-time feedback ofthe scribing by applying digital ink indicating a scribed line on thegraphical user interface; select a first item of the plurality of itemsbased on a position of the scribing within the graphical user interface;recognize a trail of the scribed line; in response to the trail of thescribed line being recognized as a first predetermined trail, performfirst processing on the first item, the first processing correspondingto the first predetermined trail; and in response to the trail of thescribed line being recognized as a second predetermined trail, performsecond processing on the first item, the second processing correspondingto the second predetermined trail and being different from the firstprocessing.
 2. The information processing apparatus of claim 1, whereinthe scribed line overlaps two or more neighboring items of the pluralityof items.
 3. The information processing apparatus of claim 2, whereinthe processor is configured to select only one of the two or moreneighboring items as the first item.
 4. The information processingapparatus of claim 1, wherein the selecting of the first item is basedon more than one point on the scribed line.
 5. The informationprocessing apparatus of claim 2, wherein the selecting of the first itemis based on more than one point on the scribed line.
 6. The informationprocessing apparatus of claim 3, wherein the selecting of the first itemis based on more than one point on the scribed line.
 7. The informationprocessing apparatus of claim 1, wherein the selecting of the first itembased at least in part on a portion of the scribed line other than astart point of the scribed line.
 8. The information processing apparatusof claim 2, wherein the selecting of the first item based at least inpart on a portion of the scribed line other than a start point of thescribed line.
 9. The information processing apparatus of claim 3,wherein the selecting of the first item based at least in part on aportion of the scribed line other than a start point of the scribedline.
 10. The information processing apparatus of claim 1, wherein theselecting of the first item is based at least in part on a middleportion of the scribed line.
 11. The information processing apparatus ofclaim 2, wherein the selecting of the first item is based at least inpart on a middle portion of the scribed line.
 12. The informationprocessing apparatus of claim 3, wherein the selecting of the first itemis based at least in part on a middle portion of the scribed line. 13.The information processing apparatus of claim 1, wherein the processoris configured to select, as the first item, an item that has a largestamount of overlapping portion overlapping the scribed line.
 14. Theinformation processing apparatus of claim 1, wherein the plurality ofitems are a plurality of visible, selectable graphical user interfacecomponents.
 15. The information processing apparatus of claim 14,wherein the plurality of graphical user interface components are optionsfrom which the user can make a selection.
 16. The information processingapparatus of claim 1, wherein the first processing is setting a firstparameter corresponding to the first predetermined trail to the firstitem.
 17. The information processing apparatus of claim 16, when settingthe first parameter to the first item, provide the user with apredetermined visual feedback indicating the first parameter.
 18. Theinformation processing apparatus of claim 17, wherein each of theplurality of items includes a text string, and wherein the predeterminedvisual feedback is applied on the text string of the first item.
 19. Theinformation processing apparatus of claim 1, wherein the secondprocessing is setting a second parameter corresponding to the secondpredetermined trail to the first item.
 20. A non-transitory computerreadable medium storing a program that causes a computer to executeinformation processing, the information processing comprising: causing adisplay to display a graphical user interface comprising a plurality ofitems; receiving an input from a user, the input being scribing over anyof the plurality of items displayed on the display; providing the userwith a substantially real-time feedback of the scribing by applyingdigital ink indicating a scribed line on the graphical user interface;selecting a first item of the plurality of items based on a position ofthe scribing within the graphical user interface; recognizing a trail ofthe scribed line; in response to the trail of the scribed line beingrecognized as a first predetermined trail, performing first processingon the first item, the first processing corresponding to the firstpredetermined trail; and in response to the trail of the scribed linebeing recognized as a second predetermined trail, performing secondprocessing on the first item, the second processing corresponding to thesecond predetermined trail and being different from the firstprocessing.